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4D/RCS  Reference  Model  Architecture 
for  Unmanned  Vehicle  Systems 

Developed  by  NIST  for  ARL  Demo  III  Program 
Adopted  by  GDRS  for  FCS  Autonomous  Navigation  System 
Adopted  by  TARDEC  for  Vetronics  Technology  Integration 

•  Hierarchical  structure 
of  goals  and  commands 

•  Representation  of  the  world 
at  many  levels 


•  Planning,  replanning, 

and  reacting  at  many  levels 

•  Integration  of  many  sensors 

stereo  CCD  &  FLIR, 
LADAR,  radar,  inertial, 
acoustic,  GPS,  internal 
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Basic  Intelligent  System 


NIST 


Perception  establishes  correspondence  between 
internal  world  model  and  external  real  world 
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Intelligent  System  Architecture 

4D/RCS  ooda  Reference  Model 
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Tasks  relative  to  nearby  objects 
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OPERATOR  INTERFACE 


A  4D/RCS  Computational  Node 


4D/RCS 


INPUT 


STATUS  COMMANDED 

ACTIONS  (SUBGOALS) 
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■  Near  Term  Prospects 


imist 


The  4D/RCS  methdology  promises  the  ability 
for  manned-unmanned  collaboration 
to  perform  tactical  behavior 


Company  level  -  30  to  40  vehicles 
Platoon  level  -  8  to  10  vehicles 
Section  level  -  2  to  4  vehicles 
Vehicle  level  -  single  vehicle 


4D/RCS  for  FCS 


24  hr  plans 
replan  every  2  hr 

5  hr  plans 

replan  every  25  min 

1  hr  plans 
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An  Example  Scenario 


NIST 


MANNED 


Scout  section  conducting  a  route  reconnaissance 


HMMWV  reconnoitering  the  right  flank  encounters  an  unexpected  water 
obstacle 

Center  HMMWV  encounters  a  bridge 

The  two  vehicle  commanders  report  their  findings  to  the  section  leader 


The  section  leader  then  commands  the 
overwatch  positions  for  near-side  security 


manned  vehicles  to  take  up 


The  section  leader  commands  the  UAV  to  look  for  a  route  around  the  water 
obstacle  UAV  sends  hi-resolution  color  images  data  back  to  the  section 
leader  for  manual  viewing,  and/or  by  scanning  the  ground  with  a  LADAR  to 
assess  the  topography 

Once  a  potential  by-pass  to  the  marsh  is  located,  the  UAV  is  commanded 
to  search  the  far  side  of  the  marsh  and  the  region  beyond  the  next  terrain 
feature  for  evidence  of  enemy  forces 

UGV  is  then  commanded  to  proceed  through  the  bypass  and  establish  an 
overwatch  position  on  the  far  side  of  the  next  terrain  feature 

The  UGV  path  can  be  automatically  generated  from  the  data  returned  from 
the  UAV  and  approved  by  the  section  leader  before  being  executed 

Once  the  UGV  is  set  in  position,  the  UAV  continues  scanning  for  enemy 
activity  further  along  the  route 

Manned  elements  perform  manual  reconnaissance  of  the  marsh  by-pass, 
and  assess  the  load  carrying  capacity  of  the  bridge. 


MANNED 
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Software  Methodology 
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large  feathery  panicles  (elongated 
clusters  of  tan/white/purple  flowers 


WaterCoveredLand 


ExtensiveMarshVegetation 
IVtajOrui  UUI  !UL/<ai  UI  i  ridlion 
WaterDepthToSixFeet 


<  Marsh  Detected 


DETERMINE  SENSOR 
PROCESSING  REQUIREMENTS 
AND  RESOLUTIONS 


>ulrushes  -  tall  tan-to-green  stems,  with 
dark  brown  cylindrical  seed  heads 
that  explode  into  white  down,  long 
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IDENTIFY  OBJECTS  and 
THEIR  RELEVANT  ATTRIBUTES 
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TASK  ANALYSIS 
to  Create  Task 
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ML 


ColumnOrg 


ControlPoints  Formantions, 


Task  Vocabulary  at  Each  Echelon 
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Hierarchy  of  Agents 
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papyrus,  narrow  green  to  tan  grass- 
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tan-to-yellow-to-white  flowers. 

Reeds  -  tall,  woody,  thin,  round,  hollow 
jointed  (tan-to-green)  stem  plants, 
long  narrow  green  blade  leaves, 
large  feathery  panicles  (elongated 
clusters  of  tan/white/purple  flowers 

pi''- 

Bulrushes  -  tall  tan-to-green  stems,  with 
dark  brown  cylindrical  seed  heads 
that  explode  into  white  down,  long 
flat  green  sword  shaped  leaves, 
cattails. 

igreen 


MostlyT  rees,SomeBushes 

ExtensiveWaterSurfaceVisible 

SlowMovingWaterCoveredLand 

SignificantTractionSlip 

MajorGroundDeformation 


StagnantWater 

OrganicMaterialOnWaterSurface 
IndeterminantGroundLevel 
Mosses, Evergreens, andShrubs 
SignificantTractionSlip 


WaterCoveredLand 
'•  ■ - 

ExtensiveMarshVegetation 
Majoroi  uui  ■vii-'ciui  1 1  id  non 
WaterDepthT  oSixFeet 
TractionSlip 


SwampDetected 


BogDetected 


MarshDetected 


IDENTIFY  OBJECTS  and 
THEIR  RELEVANT  ATTRIBUTES 


Ob, 

Features  an 

Segmented  Groupings 


FlowingWater 

NarrowWidth.IndeterminantLe'' 

NonVegetatedWaterlnMidcr 
ErodedEarthEmbankme/ 

DETERMINE  ANTECEDENT 

LargeSurfaceAreaOf 

SXS2  WORLD  STATES 

\ BoundedBySwampi 

\wingWater  \ 
jnificantWidth.Indetk 


Objects  &  Attributes 


WaterSheenOnGroundSurface 
RuttedWithStandingWater 
LittleToNoVegetation 
SignificantTractionSlip 
SignificantGroundDeformation 

World  States 


MudDetected 


Situations 


ColorCameras  LADAR 
Radar  Stereo  FUR  Nav 

SENSORY 

PROCESSING 


WORLD  MODEL 
KNOWLEDGE 


Krasnow  Institute  for  Advanced  Studies  --  George  Mason  University 


16 


iviisr 


This  is  a  tedious  process. 

There  are  many  tasks  in  the  command  library  at  each  level 

There  are  many  parameters  for  each  task 

There  are  many  objects  that  must  be  recognized 

There  are  many  situations  that  must  be  understood 

But,  the  numbers  are  not  infinite.  They  are,  in  fact, 
quite  modest.  (One  of  the  advantages  of  hierarchies.) 
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Software  Methodology 
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for  Autonomous  On-Road  Driving 

Estimated  numbers: 

~  200  tasks 
~  100  parameters 
~  1000  transition  conditions 
~  10,000  objects  or  events 

Other  skills  may  require  similar  numbers 
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RCS  Software  Develonment 


iviisr 


Cognitive  reasoning  capabilities,  planning, 
and  control  will  enable  useful  tactical  behaviors 
on  the  battlefield  within  a  decade 

The  remaining  tall  pole  in  the  tent  is  perception. 

Intelligent  machines  cannot  achieve  human 
levels  of  performance  until  machine  vision 
systems  can  perform  as  well  as  human  vision 
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Current  and  Near  Term 
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Current  and  Near  Term 


Sensor  technology  is  well  developed  - 
Color,  FLIR,  Night  Vision,  and  LADAR 

What  is  lacking  is  the  ability  to  perceive  and 
understand  situations  and  relationships 

Machine  vision  remains  far  inferior  to  human 
capabilities  in  scene  understanding  and 
situation  assessment 
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A  solution  may  lie  in 
Reverse  Engineering  the  Brain 

Cited  as  a  Grand  Challenge  by 
National  Academy  of  Engineering 

Subject  of  a  Decade  of  the  Mind  Conference 

at  Sandia  National  Labs 
Jan  13-15,2009 

http://www.dom4.org/ 

Recent  progress  in  neuroscience  &  intelligent  systems 
engineering  makes  this  a  promising  approach 
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Long  term  potential 
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Reverse  Engineering  the  Brain 

Brain  imaging  is  revealing  many  of  the  functional 
operations  in  the  brain 

Neural  modeling  is  explaining  many  of  the 
computational  processes  in  the  brain 

Computers  are  approaching  the  computational 
power  of  the  brain 

Details  of  the  functionality  and  connectivity  within 
and  between  areas  in  the  brain  are  emerging 
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Circuit  diagram 
of  visual  system 
in  brain 

12  layers 
32  areas 

Each  area  is  an  array  of 
Cortical 

Computational  Units 
(CCUs) 


cn 


Retina 
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Functional  diagram  of 
CCU  arrays  in  V1-V5 
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Long  term  potential 


Detailed  circuitry  and  functionality  of  CCU  arrays 
in  visual  cortex  are  being  understood 

About  1  million  CCUs  in  human  brain 
About  200,000  CCUs  in  visual  cortex 

About  10,000  neurons  in  each  CCU 

Connectivity  within  and  between  CCUs 
are  being  mapped 

Inputs  and  outputs  are  being  discovered 
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A  Theoretical  Model  of  a  CCU 
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windowing,  segmentation,  grouping,  computing  group  attributes  & 
state,  filtering,  classification,  setting  and  breaking  relationships 

Krasnow  Institute  for  Advanced  Studies  --  George  Mason  University  26 


M 


ecu 

Grouping 

Hierarchy 


Pointers  link 
symbols  to  pixels 
&  vice  versa 

Provide  symbol 
grounding 

Pointers  reset  every 
saccade  ~  150  ms 


V4  -  MT  groups 
level  4  CCU  array 


V3  objects 

S)  level  3  CCU  array 


V2  surfaces,  boundaries 
level  2  CCU  array 


'""-V  VI  edges,  blobs 
)  level  1  CCU  array 
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Long  term  potential 


*  Functional  modeling  of  the  human  brain  at  the  level 
of  Cortical  Computational  Units  (CCUs)  appears 
within  the  capacity  of  current  supercomputers 

*  Human  level  performance  in  perception,  situation 
awareness,  reasoning,  planning,  and  behavior 
may  be  feasible  on  desktop-class  machines  within 
two  decades 
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Summary 


Mid  term  solution  to  building  intelligent  robots 
lies  in  the  4D/RCS  reference  model 


1.  Bridges  the  gap  between  Artificial  Intelligence  (AI) 
and  modern  control  theory 

2.  Well  documented  and  tested  in  a  number  of  applications 

3.  Mature  with  many  software  development  tools 
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Summary 


Long  term  solution  to  building  robots  with 
human  level  performance  may  lie  in 
Reverse  Engineering  the  Brain 

1.  Neurosciences  and  brain  imaging  enable  visualization 
of  brain  activity  during  perception,  thinking,  planning, 
and  acting 

2.  Testable  models  of  computation  and  representation 
in  the  brain  are  emerging 

3.  Functional  modeling  of  the  human  brain  seems  within  the 
capacity  of  supercomputers  today  &  desktop  computers 
within  two  decades 
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Analogous  to  where  nuclear  physics  was  in  1908 

•  Fundamental  processes  are  understood  in  principle 

Perception  Brain  structure  and  function 

World  modeling  Cognitive  &  control  architectures 

Reasoning  Computational  equivalence 

Planning  Language 

Control  Learning  &  memory 

•  Technology  is  emerging  to  conduct  definitive 
experiments 

•  Significant  military  and  economic  applications 
will  develop  early  in  the  century 
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We  are  at  a  tipping  point 
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Impact  on  Military  Strength 


Intelligent  weapons  systems  will: 

•  outperform  manned  systems 

•  cost  less  to  train 

•  cost  less  to  maintain  readiness 

•  keep  soldiers  out  of  harm’s  way 


Intelligent  weapons  will  revolutionize  warfare 
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Contact  information 


James.Albus@gmail.com 
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